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ABSTRACT 


The  complexity  of  current  computer  systems  strongly  inhibits  management 
usage  of  analytic  modeling  techniques  in  seeking  ’good’  computer  system 
performance.  As  a result,  heuristically-based  approaches  are  in  order. 
Implementation  of  such  an  approach  requires:  (1)  a means  for  determining  the 
performance  of  a given  computer  system  processing  a given  collection  of  jobs  in 
accord  with  a specified  schedule,  (2)  a means  for  achieving  an  improved  system 
from  a given  system,  and  (3)  a technique  for  determining  when  to  stop  this 
iterative  process.  The  objective  of  this  paper  is  to  describe  an  analytically  driven 
■;  approach  to  computer  system  performance  prediction  which  can  achieve  execution 

^ speeds  of  approximately  two  orders  of  magnitude  faster  than  real  time  for 

i production  batch  installations  while  providing  detailed  informtion  on  device 

utilizations  and  delays.  Thus,  it  provides  a basic  required  capability  for  the 
development  of  heuristic  approaches  to  computer  system  performance 
i improvement. 
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INTRODUCTION 


J. 


Sstisfaction  of  the  organizational  information  processing  requirement  can  be 
achieved  through  an  iterati  c process  involving  requirements  specification  and 
system  design  entailed  by  a given  specification.  Because  of  its  obvious 
importance,  requirements  specification  has  received  considerable  study  (an 
excellent  survey  is  provided  ,n  [COUGJ  73])  and  semi-automated  procedures  for 
checking  consistency  and  completeness  [TEICD  71]  have  been  developed. 

Computer  system  design  fo”  a given  requirements  specificatiCii  has  also 
proven  to  be  a difficult  problem  for:  vendors,  the  organizational  information 
processng  function,  the  computing  center  manager  and  the  user.  Much  of  the 
available  literature  concerned  with  this  topic  is  found  under  the  general 
classification  of  performance  analysis  and  much  of  this  literature  in  turn,  is 
directed  towards  the  vendor.  In  part,  .his  focus  appears  to  reflect  the  substantial 
leverage  implicit  in  the  development  of  vendor  oriented  performance  tools. 


Development  of  computer  system  design  procedures  appropriate  to  the 
computer  center  manager  also  appears  to  have  significant  leverage.  This  topic 
can  be  structured  into  three  components;  vendor  selection,  computer  system 
sizing,  and  computer  system  tuning  [lUCAH  71],  Since  vendor  selection  is 
substantially  dependent  upon  considerations  oiher  than  minimum  cost 
determination  of  i'  system  appropriate  to  stipulated  requirements  (such  as 
maintenance,  compatibility,  etc.),  our  usage  of  the  term  system  design  will  be  in  the 
context  of  sizing  and  tuning.  Clearly,  methodologies  which  improve  sizing  and 
tuning  capabilities  provide  expanded  support  for  vendor  selection. 

Computer  system  design  methodologies  oriented  to  the  computer  center 
manager  are  required  for  two  purposes:  (i)  determination  of  the  reasonableness 
of  existing  performance,  i.e.,  ’glitch’  detection,  and  (2)  evaluation  of  the  effects  of 
system  modifications  and  alterations.  Although  vendor  oriented  results  are  of 
general  interest,  very  tight  constraints  are  implicit  in  the  requirement  for 
determ, ining  the  performance  of  a given  s/stem  executing  a specified  workload  in 
accord  with  a clearly  defined  schedule.  This  constraint  mitigates  against  the 
utility  of  a parametric  (subsystem)  approach  and  argues  for  the  development  of  a 
coherent,  coordinated  (system)  approach  integrating  data  gathering,  data 
transformation  (including  performance  prediction),  and  information  display  and 
archival. 

Development  of  a system  desigi  methodology  based  on  algorithmic  approaches 
is  precluded  by  the  cor  alexity  ot  the  computing  center  environment.  Instead, 
development  of  heuris^  . (informed  tr,al  and  error)  system  design  approaches  is 
required.  The  ability  of  heuristic  approaches  to  effectivsiv  solve  complex 
problems  is  apparent  from  the  quality  of 'current  computer  chess  playing 
programs.  Efficiency  must  also  be  considered  in  investigating  systems  design. 
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Heuriitic  System  Design  Requirements 

Development  of  a heuristic  approach  to  computer  system  design  requires  three 
capabilities: 


1.  Performance  Prediction  to  determine  the  exact  performance  of  a 
given  system  processing  a specified  workload  in  accord  with  a given 
schedule. 

2.  System  Comparison  to  determine  the  more  desirable  of  two  system 
alternatives  in  the  context  of  a given  schedule  and  workload. 

3.  Stopping  Time  Determination  for  terminating  the  iteration  cycle 
implicitly  defined  by  (1)  and  (2). 


Achievement  of  the  first  capability  requires  a very  fast  performance  prediction 
tool;  its  absence  appears  to  be  a major  reason  underlying  the  current  lack  of 
heuristic  approaches  to  system  desiga  The  second  capability  is  implicitly 
provided  by  the  output  information  generated  through  performance  prediction 
together  with  any  summary  transformations  that  may  be  employed.  One  possible 
comparison  crit'jrion  is  provided  in  [KIMBS  74B]  for  use  in  scheduling  batch 
computer  systems.  Precise  specification  of  this  information  requirss  identification 
of  the  decision  categories  available  to  computer  center  management;  a first  order 
structure  is  given  below.  Investigation  of  stopping  time  mechanisms  can  safely  be 
deferred  in  view  of  the  exploratory  nature  of  heuristic  approaches. 

The  preceding  comments  imply  that  development  of  a fast  performance 
prediction  mechanism  is  the  major  requirement  for  development  of  a Heuristic 
System  Design  (HSD)  methodology. 


Computer  Center  Management  Decision  Structure 

Development  of  a methodology  to  support  computer  center  management 
require  classifi.  ation  of  t.w  .major  decisions.  This  classification  results  in  two 
categories:  organuational  and  technological  issues.  Some  representative  issues 
are  provided  in  Table  I. 
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TABLE  I 

ISSUES  IN  COMPUTER  SYS~EMS  PERFORMA?JCE 


Organizational  Itttiet 

Technological  Ittuet 

Unnecessary  jobs 

Hardware  configuration 

Programmer  training 

Software  support 

Priority  structure 
requirements 

Job  schedules 

Online  support 
requirements 

D'jvice  schedules 

Frsctionai  system  capacity 
for  interactive  suf.pcrt 

File-Device  assignment 

Local  control  of  data 
base 

Device/Channel  ratio 

System  conipatibility 
requirements 

Extended  instruction 
set 

Control  limit  specification 

Processor  scheduling 
algorithm 

First  order  technological  decisions  can  be  divided  into  four  claisses: 

01.  hardware  alterations  and  enhancements, 

02.  software  capabilities, 

03.  job  schedules,  and 

04.  file-device  assignments. 

For  technological  issues  the  concept  of  an  optimal  system  is  reasonable  although 
elusive. 

Organizational  issues  have  two  significant  characteristics:  (1)  management 
satisfices  rather  than  optimizes  on  these  variables,  and  (2)  determination  of 
reasonable  values  for  these  variables  is  heavily  site  dependent.  Thus,  such  issues 
cannot  be  resolved  through  a generic  approach  and  must  consequently  be  attached 
on  a case  by  case  basis  in  which  the  skills  of  the  management  consultant  are  often 
required. 
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The  distinction  between  organizational  and  technological  issues  is  not  inviolate. 
For  instance,  operator  efficiency,  an  organizational  issue,  is  significantly  affected 
by  the  technological  support  provided  (advance  notification  of  tape  mounts, 
scheduling  assistance,  etc.). 

Decisionmaking  is  further  affected  through  the  presence  of  management  and 
user  constraints.  Management  constraints  are  explicitly  represented  through 
budgetary  limitations  which  implicitly  reflect  a desire  to  assure  a ’reasonable’  level 
of  system  utilization.  Presumably,  utilizations  of  lOOX  on  all  devices  would  be 
regarded  as  completely  acceptable.  User  constraints  are  reflected  in  the 
acceptable  range  of  response/turnaround  times  and  times  equivalent  to  those 
experienced  on  a stand-alone  system  would  be  regarded  as  completely  acceptable. 

A critical  function  of  computer  center  managerr«nt  is  achievement  of  an 
acceptable  system  providing  reasonable  balance  among  these  constraints.  For  a 
given  requirements  specification,  this  constitutes  the  system  design  problem  and 
its  two  components,  sizing  and  tuning. 


Paper  Ohjeetivet 

The  thesis  of  this  report  is  that  a support  methodology  appropriate  to 
computer  center  management  can  be  developed,  that  this  development  requires  a 
heuristic  approach,  and  that  the  major  implementational  requirement  for  such  an 
approach  is  the  development  of  a fast  performance  prediction  capability.  To 
support  this  thesis  we  have  provided  an  overview  of  the  computer  center 
management  decision  process  and  identified  the  major  components  of  a heuristic 
system  design  approach.  Next,  we  shall  identify  performance  prediction 
requirements  implicit  in  this  approach,  and  then  describe  one  means  for  their 
realization.  Comparisons  with  a computer  system  simulator  are  provided  and 
observations  regarding  desirable  directions  for  future  research  are  made. 
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PERFORMANCE  PREDICTION  REQUIREMENTS 


Development  of  an  HSD  supportive  performance  prediction  mechanism  requires 
identification  of:  (1)  objectives  and  information  requirements,  (2)  accuracy 
requirements,  (3)  level  of  detail,  ano  (4)  modeling  technology.  In  this  section, 
goals  for  each  of  these  categories  will  be  detailed.  The  remainder  of  the  paper 
may  be  viewed  as  a demonstration  of  the  feasibility  of  their  achievement. 


Objectivet  and  Information  Reqairementt 

Technological  control  of  a computer  system  requires  balancing  management 
utilization  objectives  with  user  response/turnaround  time  goals.  Tnis  requires 
determination  of  delay  by  job  class  and  identification  of  multiprogramming  related 
delays.  (A  turnaround  time  of  ten  hours  is  probably  acceptable  if  the  stand-alone 
processing  time  is  nine  hours  and  is  probably  unacceptable  if  the  stand-alone 
processing  time  is  ten  minutes.) 

Technological  control  decisions  are  reflected  in  the  four  decision  categories 
(0l)-{D4j  cited  above.  It  follows  that  evaluation  of  alternatives  requires 
Knowledge  of  individual  device  utilizations  and  delays.  Further,  it  is  desirable  that 
this  information  be  broken  out  on  a pe"  job  basis,  a per  shift  basis,  and  as  will  be 
seen  later,  a per  time  segment  (period  of  time  during  wnich  the  composition  of  the 
mix  remains  constant)  basis. 


Accuracy  Requiremrnlt 

Persuasive  salesmanship  has  been  used  to  convince  computer  center 
management  that  accuracy  levels  of  one  percent  are  readily  obtair.able  with 
current,  commercially  available  simulators.  Further,  feasibility  demonstrations 
show  that  such  accuracy  levels  can  be  achievec  after  some  tuning  of  the  simulator. 
The  existence  of  this  tuning  requirement  mitigates  against  the  utility  of  the 
simulator  in  investigating  new  configurations  or  in  evaluating  the  reasonableness 
of  existing  performance--two  primary  uses  for  a simulator. 

The  difficulty  of  designing  and  implementing  a simulator  is  a function  of  the 
accuracy  level  required.  The  user  of  a simulator  naturally  seeks  a very  high 
accuracy  level,  say  \1.  (that  is  jobs  - predj/obs  < .01  where  obs  denotes  the 
observed  value  of  a statistic  and  pred  denotes  the  value  of  the  statistic  predicted 
through  usage  of  the  simulator).  Acceptance  of  a lower  level  of  accuracy  requires 
education  of  the  user  through  identification  of  accrued  advantages  typically 
including:  reduction/elimination  of  the  need  for  calibration,  increased  speed  of 
execution,  and  expedition  of  the  verification/validation  process.  Based  upon 
discussions  with  managers,  it  is  the  author's  conjecture  that  most  users  can  easily 
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tolerate  an  accuracy  level  of  20Z.  This  paper  hypothesizes,  and  partially  verifies, 
that  such  an  accuracy  level  can  be  achieved  through  usage  of  fast,  analytically 
driven  techniques. 


Ltv*l  of  Detail 

Performance  prediction  techniques  can  be  classified  in  terms  of  the  basic  level 
of  detail  incorporated:  hardware  (register),  operating  system,  resource  allocation, 
time  segment  and  job.  Decreasing  the  level  of  detail  decreases  the  potentially 
achievable  level  of  accuracy  as  well  as  the  cost  of  implementation  and  cost  of 
verification/validation  of  the  simulation.  Our  objective  is  to  achieve  an  accuracy 
level  of  twenty  percent,  and  our  assertion  is  that  this  level  of  accuracy  can  be 
achieved  through  prediction  of  time  segment  performance  and  aggregation  of  the 
resulting  statistics  to  achieve  job  and  shift  performance.  This  assertion  is 
unprovable.  However,  supporting  evidence  is  provided  in  the  comparisons 
detailed  in  Figures  1-5,  included  at  the  end  of  the  paper.  Although  the  207. 
discrepancy  level  is  not  achieved  in  all  cases,  non-achievement  seems  clearly  due 
to  usage  of  an  approximation  technique  providing  only  lower  bound  estimators  for 
processor  utilization.  Subsequent  efforts  directed  to  achievement  of  the  desired 
tolerance  level  through  development  of  improved  processor  utilization  estimators 
seem  very  liKely  to  succeed,  fvtoreover,  it  will  be  noted  that  the  level  of 
discrepancy  decreases  as  the  level  of  processor  utilization  increases. 
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Modeling  Technology 

Prediction  of  system  performance  can  be  approached  through  either  analytic 
or  simulation  techniques.  Because  of  the  limited  information  available  from  an 
individual  analytic  tool,  analytic  approaches  have  been  relegated  to  the  role  of 
design  tools  used  in  gaining  insight  into  subsystem  tradeoffs  and  policy. 
Additionally,  analytic  models  are  occasionally  used  to  estimate  gross  system 
performance  characteristics  in  which  extensive  aggregation  of  system  components 
is  required  to  achieve  computational  feasibility  [MOORC  71],  [HANSF  71].  The 
advantage  of  analytic  approaches  is  their  speed  of  implementation  and  execution 
coupled  with  relatively  low  cost  for  developed  models;  the  disadvantages  include 
extensive  simplifying  assumptions,  the  feasibility  of  incorporating  only  a meager 
level  of  detail,  the  difficulty  of  understanding  them  without  extensive  training  in 
modeling  techniques,  and  the  prevailing  lack  of  reasonable  user  interfaces. 

Simulation  approaches,  in  contrast,  permit  incorporation  of  almost  any  desired 
level  of  detail.  Since  there  is  a prevailing  tendency  to  incorporate  superfluous 
factors,  and  a great  discrepancy  between  the  rate  at  which  events  .xcur  in  a 
computer  system  (micro  seconds— milli  seconds)  and  the  run  time  of  a computer 
system  (hours— days),  most  simulation  models  execute  relatively  slowly,  e.g.,  1-10 
times  faster  than  real-time. 
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Analytic  and  simulation  approaches  can  be  regarded  as  two  endpoints  of  a 
spectrum  of  possible  approaches  to  computer  system  performance  prediction. 
Intermediate  points  within  this  spectrum  would  blend  simulation  and  analytic 
approaches  in  a desire  to  balance  execution  speed,  output  information,  required 
input  data,  and  accuracy.  The  objective  of  this  paper  is  to  Hemonstrate  that 
through  proper  development  of  an  analytically  driven  approach,  information 
essentially  equivalent  to  that  provided  through  current,  commercially  available 
simulators,  can  be  achieved  very  quickly.  Thus,  the  basic  requirement  for 
implementation  of  a heuristic  approach  to  computer  system  design,  sizing,  and 
tuning  will  have  been  achieved. 
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ANALYTIC  PREDICTION  OF  SYSTEM  PERFORMANCE 


Development  of  an  analytic  technique  for  predicting  computer  svstem 
performance  requires  identification  of:  (1)  the  precise  analytic  techniques  to  be 
used,  (2)  the  input  data  required,  (3)  the  output  information  sought,  and  (4)  the 
techniques  to  be  used  to  derive  (3)  from  (2).  Item  (4)  proves  not  to  be 
straightforward  since  a variety  of  analytic  techniques  must  be  interrelated  to 
obtain  a capability  which  can  provide  output  information  competitive  with  that 
provided  by  a resource  allocation  level  simulator.  Each  of  these  issues  will  now 
be  discussed  in  turn. 


Selsction  of  Baiic  Analytic  Approach 

Two  major  approaches  to  modeling  processor-l/0  interaction  within  a computer 
system  can  be  differentiated  through  their  representation  of  I/O  delay.  The 
queuing  network  approach  represents  I/O  devices  by  their  service  time  and 
obtains  global  system  results  (device  utilizations  and  delays)  in  terms  of  these 
service  times  and  the  transition  matrix  guiding  migration  among  I/O  devices  and  the 
processor  (also  represented  in  terms  of  its  service  time).  Alternative  analytic 
approaches  represent  I/O  devices  in  terms  of  total  I/O  delay  and  are  provided  by 
the  machine  repair  model  and  an  approximation  technique  termed  the  system 
orocess  model  [KIMBS  75]. 

Wide  usage  of  the  queuing  network  approach  has  been  made  in  computer 
network  analysis  [KLEIL  64],  communications  system  design  fKLEIL  70],  and 
computer  systems  analysis  [BASKF  71],  [BUZEJ  71]  and  [MOORC  71],  The  results 
obtained  via  this  approc-li  have  proved  encouraging  and  extensive  research  is 
' continuing  on  a variety  of  related  topics.  Usage  of  this  approach  as  the  kernel  of 
an  analytically  driven  performance  prediction  mechanism  proves  difficult  for  two 
roasons:  (1)  representation  of  data  path  delays,  and  (2)  computational  complexity. 
The  first  reason  reflects  the  fact  that  incorporation  of  data  paths  requires  a 
methematical  capability  to  handle  queuing  networks  with  correlated  service  times  - 
a capability  which  currently  appears  to  be  in  advance  of  the  state  of  the  art.  The 
second  reason  reflects  the  fact  that  the  complexity  of  the  queuing  network 
approach  tends  to  increase  with  the  square  of  the  number  of  nodes  incorporated. 
Thus,  published  studies  have  found  lumping  of  nodes  necessary  to  reduce  the 
dimensionality  from  the  fifty  or  more  resources  comprising  the  typical  large  scale 
computer  system  to  5-10  [MOORC  71].  However,  some  work  has  appeared  which 
is  concerned  with  more  computationally  efficient  procedures  [BU7EJ  73]. 

In  I/O  delay  approaches,  the  input  information  is  mean  processor  burst  and 
mean  I/O  delay;  the  output  information  is  processor  utilization  and  delay.  Vhe 
standard  machine  repair  model  assumes  exponentially  distributed  variables  and  an 
iterative  technique  has  been  developed  [CAVED  67]  for  calculating  the  relevant 
statistics  under  the  assumption  of  non-exponentially  distributed  I/O  delays.  This 
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technique  has  been  successfully  applied  to  computer  system  modeling  in  which  the 
workload  is  characterized  by  an  ’average’  job(s)  FSEKIA  72]. 

The  system  process  model  [KIMBS  75]  uses  a similar  job  characterization. 
Exponential  assumptions  are  shown  to  constitute  a natural  midpoint  for  a wide 
range  of  distributional  assumptions.  The  estimator  of  processor  delay  is 
independent  of  the  approximation  used  to  obtain  processor  utilization;  thus, 
alternative  techniques  for  estimating  processor  utilization,  includirg  the  machine 
repair  approach,  can  be  used.  This  computational  technique  hjs  been  used 
because  of  the  straightforward  nature  of  the  calculations  which  constitute  the 
kernel  computation  used  in  developing  an  analytically  driven  performance 
prediction  technique  termed  ASIN/I. 

I/O  delay  approaches  naturally  enable  a hierarchical  approach  to  calculation  of 
system  statistics  in  which  1/0  delay  is  first  determined  followed  by  evaluation  of 
other  system  performance  characteristics.  Since,  as  shown  in  the  following 
section,  the  kernel  calculation  for  I/O  delay  and  processor  utilizat'On  is  relatively 
straightforward,  the  complexity  of  the  total  computation  is  approximately  linear  in 
the  number  of  devices  comprising  the  system,  and  is  thus  competitive  with 
simulation  approaches  for  both  large  and  small  systems. 
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DESCRIPTION  OF  AN  ANALYTICALLY  DRIVEN 
PERFORMANCE  PREDICTION  TOOL  (ASIM) 


The  major  objective  in  the  development  of  ASIM  was  fast  prediction  of  the 
performance  of  a given  system  executing  a specified  collection  of  jobs  in  accord 
with  a defined  schedtie.  in  this  section  we  describe:  (1)  simplifying  assumptions 
used  in  the  initial  ASiM  implementation,  (2)  input  data  characterization,  (3)  output 
information  developed,  (4)  conceptual  steps  involved  in  obtaining  system 
performance  statistics,  and  (5)  accurac'  /timing  considerations. 


ASIM  Implementation  Atsumptions 

The  following  eight  simplifying  assumptions  were  chosen  to  represent  a 
compromise  between  code  simplification  and  demonstrating  feasibility  appropriate 
to  actual  utilization  in  a computing  environment. 


1.  A single  processor  system. 

2.  One  data  path  per  device. 

3.  Zero  setup  times  for  jobs  (both  initiation/termination  and  disk/tape 
mount/dismount  times). 

4.  All  jobs  have  equal  priority. 

5.  One  active  data  set  per  device  per  job. 

6.  No  unit  record  equipment. 

7.  Jobs  use  a fixed  amount  of  user  memory. 

8.  A maximum  of  thirty  drums,  disks,  and  tapes;  all  drums  are  shared,  a 
user  specified  number  of  disks  may  be  shared,  and  all  tapes  are 
non -shared. 


a 


1 


I 


i 
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Removal  of  the  single  processor  assumption  is  possible  at  the  relatively  minor 
cost  of  requiring  another  (internal)  list  for  each  additional  processor  and 
incorporation  of  an  estimator  for  processor  contention  impact  on  system 
performance.  Explicit  incorporation  of  data  paths  requires  development  of  a 
suitable  approach  to  representation  of  data  path  induced  delay;  existing  models  in 
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the  literature  are  oriented  to  static  prediction  and  inappropriate  to  the  objectives 
of  this  model.  Removal  of  the  remaining  assumptions  is  a programming  exercise 
(note  that  average  working  set  size  can  be  used  for  virtual  memory  systems  in 
place  of  partition  size  for  those  systems  based  on  a working  set  philosophy). 


ASIM  Input  Data 

The  input  data  to  ASIM  consists  of  three  files:  (It  Job/System  Descriptions,  (2) 

System  Charo  rsristics,  and  (3)  Switch  Settings.  Sv/itch  settings  constitute  a 
superset  of  the  standard  GASP  switch  settings  [PRITA  69]  which  initialize  a run 
and  will  not  be  discussed  further. 

Table  II  indicates  the  components  of  a Job/System  Description  (JSD).  As  is 
apparent,  a JSD  is  related  to  a synthetic  module  characterization  of  a job 
[BUCHW  69],  [HAMF  73],  [SREEK  74]  and  consists  of  two  components.  The 
exogenous  component  describes  the  environment  defined  for  the  job  and  its 
relation  to  other  jobs,  while  the  endogenous  component  describes  the  resource 
requirements  and  utilization  rates  for  an  individual  job.  Although  automatic 
generation  of  JSD’s  is  not  currently  feasible,  careful  examination  of  the  more 
sophisticated  accounting  systems  demonstrates  the  feasibility  of  capturing  these 
data  through  minor  modifications.  Indeed,  SMP  data  is  currently  bein.g  utilized  to 
generate  input  data  for  some  currently  available  simulators  [EDGEG  73]. 

It  should  be  noted  that  the  JSD  was  so  nan.ed  since  it  constitutes  a viable 
description  of  the  job  only  in  the  context  of  a given  target  system.  In  particular, 
no  assertion  is  intended  that  the  same  source  code  executed  on  systems  produced  j 

by  two  different  vendors  will  be  the  produce  the  same  JSD’s.  ' 

Jobs  are  assumed  to  be  initiated  in  the  order  indicated  by  the  list  of  JSD’s  I ' 

subject  to  three  constraints;  precedence  satisfaction,  arrival  time  satisfaction  (a 
job  car-iot  be  initiated  prior  to  arrival),  and  resource  availability.  If  one  of  these 
constraints  is  unsatisfied,  initiation  is  deferred  pending  its  satisfaction.  Further, 
no  attempt  is  made  to  look  ahead  in  the  JSD  sequence  to  determine  if  another  job 
could  be  initiated.  It  should  be  noted  that  this  requirement  is  necessary  in  order 
to  permit  study  of  computer  system  performance  as  a function  of  the  schedule 
[KIMBS  74B].  It  does  not  reflect  current  job  scheduling  in  which  both  the 
operating  system  and  the  external  scheduling  mechanism  jointly  share 
responsibility.  Although  the  current  system  scheduling  approach  has  proven 
useful  (perhaps  unavoidable),  in  practice  it  greatly  impedes  determination  of  the 
performance  of  a given  computer  system  executing  jobs  in  accord  with  a 
prescribed  schedule.  Accordingly,  the  approach  which  we  have  described  has 
been  implemented.  Modification  of  this  approach  to  more  accurately  reflect  the 
characteristics  of  scheduling  as  implemented  on  an  individual  computer  system  is 
essentially  a programming  problem. 


TABLE  II 


JOB/SYSTEM  DESCRIPTION  (JSD)  COMPONENTS 


F^xogenout  JSD  V<  tabUt 


Name 

Time  of  Arrival 

Stat'c  Priority 

Job  Due  Date 

Job  Setup  Time 

Job  Precedence  Relations 

Number  of  Job  Steps 


Fndogcttout  JSD  Ftrinhles 


Processor  Interaction 

1.  Job  Processor  Requirements 

2.  Processor  Burst  Description 

3.  CPU/I/O  Overlap 


Memory  Requirements 


File  List 

1.  Number  of  Files  Accessed  by  Job 

2.  File  Names 

3.  File  Access  Count 

4.  Data  Transfer  Size 


File  Characteristics 
(For  Each  Listed  File) 

File  Device  Location 

2.  Latency 

3.  Seek  Time  Distr.oution  (for  disks) 

4.  File  Size 

5.  File  Organization 
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TABLE  III 

SYSTEM  CHARACTERISTICS 

Mjmber  of  Drums  0* 

Number  of  Disk  Drives  o 

Number  of  Tape  Drives  0 

Number  of  Non-Shared  Disk  Drives  0 

Amount  of  Core  Memory  100 

Rotation  Time  of  Drum  35.0000 

Number  of  Pages  per  Drum  Track  6 

Rotation  Tim.e  ot  Disk  16.66G7 

Number  of  Pages  per  Disk  Track  5 

Total  Number  of  'Tvlinders  All 

Minimum  Seek  Time  10.0000 

Maximum  Seek  Time  55.0000 

Average  Seek  Time  30.0000 

Tape  Interrecord  Gap  0 

Cost  Per  Unit  T'me  of  CPU  10.000000 

Cost  of  Core  Ptr  Unit  Time  1.000000 

Cost  of  Drum  Per  Unit  Time  0.015000 

Cost  of  Disk  Per  Unit  Time  0.006800 

Cost  of  Tape  pe»'  Unit  Time  0.005000 

♦Numbers  are  representative  values  used  in  the  example. 


/iSlM  Output  Information 

Output  information  generated  by  ASIM  is  aggregated  into  three  categories:  (1) 
job  performance  statistics,  (2)  shift  performance  statistics,  and  (3)  time  segment 
performance  statistics.  The  information  in  each  category  consists  of  device 
utilizations  and  delays  as  shown  m Table  IV  together  with  some  header  information 
which  is  not  given.  For  a job  this  header  information  includes  job  identification 
and  for  a time  segment  the  collection  of  jobs  in  concurrent  execution  is  displayed. 


TABLE  IV 


ASIM  OUTPUT  INFORMATION 


Processor  Utilization 
Processor  Delay 

Memory  Utilization 

Average  Degree  of  Multiprogramming 
Average  I/O  Delay 
Average  System  Utilization 

Individual  Drum  Utilization 
Individual  Drum  Delay 
Individual  Drum  Service  Time 

Individual  Disk  Utilization 
Individual  Disk  Delay 
Individual  Disk  Service  Time 

Individual  Tape  Utilization 
Individual  Tape  Delay 
Individual  Tape  Service  Time 


/JS/Af  Performance  Calculations 

Let  an  event  denote  the  time  at  which  either  a job  initiation  job  termination 
occurs  and  let  a time  segment  denote  the  time  between  two  successive  events. 
For  a shift  in  which  N jobs  are  processed,  there  will  be  at  most  2N  events  and 
2N-1  time  segments;  the  uoper  bound  is  reached  for  non-concurrent  initiations. 
Since  the  composition  of  the  mix  is  constant  over  a time  segment,  analytic 
prediction  of  time  segment  performance  is  in  order.  Standard  aggregation 
teciiniques  used  for  st;tistics  accumulation  in  simulation  can  then  be  used  to 
develop  job  and  shift  performance.  Additionally,  time  segment  performance  is 
captured  since  poor  performance  for  a job  implies  poor  performance  for  at  least 
one  time  segment.  Further,  development  of  heuristic  scheduling  pre-edures 
requires  knowledge  of  time  segment  performance  characteristics  [KIMBS  74B]  to 
evaluate  the  desirability  of  alternative  job  interchanges  in  heur'stically  seeking  a 
’good’  schedule. 

Label  the  collection  of  jobs  to  be  processed  during  a representative  time 
segment  Jl,...,Jn.  Generation  of  processor  delay  and  utilization  requires 
generation  of  an  average  active  and  blocked  interval  for  this  time  segment 
appropriate  to  *his  collection  of  jobs. 
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Determination  of  average  I/O  delay  requires  care.  In  its  calculation,  an 
av^arage  processor  burst  which  is  simply  the  arithmetic  average  of  the  processor 
bursts  for  jobs  Jl,...,Jn,  is  used.  It  is  evident  from  the  Job/System  Descriptions 
that  knowledge  of  the  JSD’s  for  J’.,...,Jn  permits  straightforward  calculation  of  the 
probability  of  a reference  to  an  individual  device  during  the  time  segment 
(assuming  accesses  are  equally  probable  over  a time  segment).  This  probability, 
together  with  the  average  processor  burst  described  above  permits  a 
mathematical  determination  of  upper  and  lower  bounds  for  the  i ,:erarrival  times  to 
a given  device.  (Determination  of  the  mein  interarrival  time  pr.jves  difficult  since 
it  is  affected  by  device  characteristics  and  queues;  thus,  its  determination  requires 
an  iterative  approach  and  was  deferred  in  the  interest  of  computational  speed.) 
Given  these  bounds,  bounds  for  device  delay  can  then  be  obtained  through 
queuing  calculations. 

Usage  of  available  device  models  within  the  literature  is  a natural  objective 
which  is  precluded  by  their  assumption  of  an  infinite  calling  population.  In 
practice,  for  batch  systems,  a degree  of  multiprogramming  in  the  range  of  3-6  has 
been  typical  [RODRJ  72].  Further,  the  number  of  ’batch  equivalents’  in  interacti  ve 
systems  has  been  observed  to  fall  in  this  range  since  an  average  batch  job 
constitutes  approximately  the  same  system  load  as  ten  average  interactive  jobs 
[MOORC  71].  Clearly,  such  estimators  are  at  bes*  crude.  Moreover,  data  on 
batch  equivalents  for  non-university  environments  is  not  readily  available  although 
similar  computational  approaches  can  be  used. 

Because  of  the  unsuitability  of  existing  I/O  device  models,  a technique 
described  in  [LAVES  75]  for  obtaining  the  steady  state  queuing  time  distribution 
for  the  M/G/1  finite  capacity  queue  proved  appropriate  and  is  summarized  in  the 
following  theorem: 


Theorem  I 

The  expected  system  residence  time  of  an  arriving  request  to  an  M/G/1  queue 
with  finite  npacity  N is  given  by: 

* E[SRT]  - NE[S]  - zjJ'J  k p(k,N)/m, 


where  m is  the  arrival  rate  (to  the  infinite  capacity  system),  E[S]  is  the  average 
service  time  and  <p(k,i'\l);  1 < k <.  N>  are  given  by: 

p(k,N)  - a(N-k)/A(N-l);  I < k < N, 


and  the  terms  i.-vj),  A(j)  are  determined  recursively  through: 


a(0)  = 1 
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a(l)  - b(l)/(l-b(D) 


(a(u)b(k+l-u)+a(k))/(l-a(D),  2<ksN 

A(k)  -2;'^  a(j)j  0<k<l\l 

j =G 


and 

^+00 

■ I (l-F(t))v[(mt)«(k-l)]»m»exp(-mt)/(k-l)!  dt 

^ o 

Note  that  although  data  paths  are  net  explicitly  incorporated  in  the  present 
version  of  ASIM,  their  incorporation  is  feasible  in  thij  approach  described  and  an 
approximately  linear  complexity  in  terms  of  the  number  of  de/ices  is  retained. 


Having  obtained  the  delay  for  an  individual  device,  average  delay  for  j|| 
devices  can  then  be  obtained.  This  then  permits  determination  of  processor 
utilization  U and  t.ie  processor  delay  D ' id  the  following  theorem  [KIMBS  75]: 


Theorem  2 

Let  A and  B denote  the  average  length  of  the  active  and  blocked  intervals  of 
the  process.  For  K>2  concurrently  executing  statistically  identical  processes; 


(i)  U > 1-  p ) 
(ii)  D - (K-J)AU(K) 


where  P-B/(A+B),  and  J»B/A  denotes  the  expected  number  of  active  intervals 
which  can  be  initiated  during  a blocked  interval. 

Application  of  this  theorem  and  the  I/O  results  described  e?rlier  yields  all 
device  delays  and  utilizations.  Determination  of  core  utilization  is  a simple 
calculation. 


I I 

j AStM  Timir.g  and  Implementation  Considerations 

For  portability  reasons,  ASIM  has  been  implemented  in  a Fortran  superset 
GASP  [PRITA  69].  Although  GASP  is  a simulation  language,  it  was  used  for  its 
output  formatting  and  repo-t  generation  capabilities  rather  than  its  I'st 
manipulation  capabilities.  Retrospect  suggests  that  direct  coding  of  desired  output 
capabilities  and  elimination  of  the  GASP  routines  is  feasible  and  would  ease 
portability. 

Dt^scription  of  ASIM  speed  is  misleading  since  it  is  a function  prima.  ' of  the 
number  of  jobs  executed  during  a shift  rather  than  their  duration.  Thus, 
execution  time  is  the  sum  of  the  time  segment  execution  times  which  are 
effectively  linear  in  the  number  of  distinct  devices  referenced  during  the  time 
segment.  Consequently,  overall  execution  time  is  nearly  linear  as  a function  of  the 
number  of  jobs  and  distinct  I/O  devices.  Typically,  a time  segment  performance 
calcula*  on  requires  approximately  one  second  of  processor  time  on  a medium 
sized  .omputer  such  as  TENEX.  Thus,  determination  of  performance  statistics  for 

a shift  with  50  jobs  requires  less  than  two  minutes  of  processor  time.  Although 

most  shifts  process  a sign..'icantly  greater  number  of  jobs,  many  of  these  jobs 

require  only  very  limited  amounts  of  resources  and  are  better  represented  in  the 
aggregate  rather  than  individually,  in  view  of  assumption  3. 

I Although  the  description  of  ASIM  calculations  is  relatively  straightforward  this 

reflects  hindsight  and  is  not  represented  in  the  current  version  which  consists  of 
[ approximately  1800  source  statements.  Knuth’s  remark  to  the  effect  that  the  best 

* way  to  implement  a program  is  to  code  it  once,  throw  it  away  and  code  it  again 

seems  very  appropriate.  A new  version  is  now  being  coded  and  the  objective  is 
to  reduce  the  number  of  lines  of  source  code  to  approximately  500.  Ihe  next 
section  provides  some  comp»''rison  information  for  ASIM  evaluation. 

I 
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AN  EXAMPLE 


Evaluation  of  a simulator  is  a somewhat  difficult  task.  The  literature  is  notably 
lacking  in  detailed  evaluations  and,  if  discussed  at  all,  evaluation  is  usually  given  in 
terms  of  a few  ’representative’  examples.  (Evaluation  of  commercially  available 
simulators  is  somewhat  difficult  since  most  of  them  require  ’tuning’.  By  contrast, 
ASIM  has  virtually  no  tunable  variables.) 

Evaluation  of  ASIM  is  somewhat  simpler  since  the  objective  is  to  demonstrate 
that  information  approximately  as  detailed  as  that  obtained  via  simulators  can  be 
Obtained  while  a significant  increase  in  speed  is  also  obtained.  For  comparison 
purposes,  a resource  allocation  simulator  (ISIM)  [KIMBS  74A]  was  used.  Figures 
1-5  provide  comparison  informatbn.  These  comparison  statistics  were  obtained 
for  five  identical  concurrently  executing  jobs  accessing  three  disks  in  a uniform 
manner  and  differing  only  in  the  length  of  the  average  processor  burst  which  was 
5,  10,  20,  40  and  80  ms.  Each  job  requires  1000  processor  bursts,  arrived  at 
tine  0,  had  no  precedence  or  due  date  constraints,  and  was  sufficiently 
undemanding  in  core  requirements  to  permit  simultaneous  initiation. 

Figure  1 indicates  that  the  c(.rrelation  between  ASIM  and  ISIM  predictions  (for 
a system  whose  characteristics  are  as  indicated  in  Table  III)  was  very  good  for 
gross  statistics  such  as  the  system  residence  time  (elapsed  time  from  initiation  to 
termination  of  job).  Further,  Figure  2 shows  a reasonable  correlation  between 
overall  system  utilization  for  ASIM  and  ISIM  as  represented  in  the  system  reward 
function  (a  dollar  weighted  device  utilization  statistic  computed  over  all  four  device 
categories).  Figure  3 demonstrates  that  the  average  processor  delay  also 
compares  well.  Figure  4 shows  that  the  comparison  between  processor 
utilizations  is  not  as  satisfactory  as  is  also  true  for  the  comparison  between  disk 
delays  as  shown  in  Figure  5.  This  discrepancy  reflects  the  fact  that  for  low 
access  rates  to  I/O  devices,  the  gap  between  the  computed  upper  and  lower  delay 
bounds  for  individual  devices  is  fairly  wide.  Indeed,  the  envelope  between  these 
two  bounds  cunldmad  the  ISIM  prediction  in  all  cases.  Thus,  a refinement  of  the 
I/O  delay  prediction  technique  is  required  and  is  perhaps  the  most  important 
improvement  needed.  A major  requirenient  for  achieving  this  accuracy 
improvement  is  developing  an  mprcved  estimator  for  p'oeessor  utilization,  as  is 
apparent  from  Figure  4. 

Collectively,  this  information  indicates  that  the  desired  level  of  accuracy  has 
been  achieved  for  gross  overall  statistics  and  that  some  refinement  is  needed  for 
the  device  statistics.  Thus,  we  argue  that  the  basic  feasibility  of  obtaining  a 20Z 
accuracy  level  has  been  shown  and  effort  directed  toward  consolidation  and 
improvement  is  merited.  It  is  natural  to  consider  the  quality  of  the  comparisons 
‘Or  non  statistically  identical  jobs.  Surprisingly,  the  compa'-isons  were  slightly 
better  and,  in  general,  the  accuracy  of  the  predictions  seems  to  improve  with 
increasing  system  complexity. 
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DIRECTIONS  FOR  FUTURE  RESEARCH 


Results  obtained  to  o 'te  support  the  hypothesis  that  analytically  driven 
performance  prediction  techni^.ies  providing  signifi.ant  speed  advantages  over 
those  obtainable  via  a simulator  can  be  constructed.  Further,  the  discrepancies 
bet’-^een  ASIM  and  ISIM  appear  due  to  inv^dequacies  in  existing  analytic  techniques 
rather  than  to  fundamental  deficiencies  in  the  approach.  Extensive  e.<,.^loration  of 
the  limits  of  this  approach  is  clearly  desirable. 

Discussion  of  ASIM  speed  is  misleading  since  it  it  effectively  independent  of 
the  length  of  the  job.  For  production  batch  environments  in  which  the  average 
job  requires  five  minutes  or  more  to  execute,  a claim  of  two  Orders  of  magnitude 
faster  than  real  time  is  reasonable.  For  a university  environment  in  *.'hich  the 
average  job  requires  two  seconds  of  processor  time  this  claim  is  clearly 
inappropriate:  thus  the  restriction  to  pro  xtion  batch  environments. 

Usage  of  ASIM  is  not  restricted  to  batch  environments  provided  that  one  is 
willing  to  represent  the  interactive  load  through  an  ’average’  approach.  Because 
the  standard  deviation  of  resource  requests  for  interactive  jobs  is  significantly 
larger  than  the  mean  [MOORC  71],  development  of  upper  and  lower  bounds  seems 
more  appropriate. 


Device  utilizations  and  delays  are  the  usual  first  orri^r  statistics  of  a stochastic 
process.  In  performance  analysis  second  order  statistics  such  as  variances  and 
correlations  are  also  known  to  be  useful  as  demonstrated  by  the  widespread 
interest  in  the  CPU/channel  overlap  statistic.  Determination  of  the  feasibility  of 
developing  such  statistics  is  a research  issue  and  should  only  be  approached  after 
channel  effects  have  been  adequately  incorporated. 

The  possibility  of  very  fast  performance  prediction  techniques  with 
information  output  compatible  with  that  provided  by  simulations  permits  essentially 
different  approaches  to  two  important  issues. 

1.  Computer  system  scheduling  is  usually  performed  on  a trial  and  error 
basis  in  view  of  the  failure  of  existing  probabilistic  and  deterministic 
techniques  to  adequately  reflect  the  complexity  of  the  computer 
operation.  Thus,  probabilistic  techniques  cannot  handle  the 
dimensionality  of  the  system  as  reflected  in  both  the  number  of  devices 
and  the  constraints  which  must  be  observed,  while  deterministic 
(mathematical  programming)  techniques  cannot  reflect  the  random 
phenomena  occuring  in  computer  systems.  Collectively,  these  factors 
indicate  the  desirability  of  a heuristic  approach  to  scheduling.  Using 
ASIM,  an  initial  feasibility  study  of  this  possibility  has  been  undertaken 
[KIMBS  74B]  and  the  results  are  encouraging. 

2.  Design  of  large  networked  systems  is  difficult  and  currently 
unsupported  by  a methodology  to  reflect  the  effects  of  different  design 
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alternatives.  Extension  cf  the  ASIM  approach  seems  to  permit 
development  of  such  a methodology  [KIMBS  74C]. 


Finally,  it  is  of  interest  to  note  that  automatic  gathering  of  ASiM  input  data  is 
feasible  with  minor  modifications  to  the  accounting  system  on  most  large  computer 
systems  [EDGEG  73].  Thus,  the  initial  requirement  for  development  of  an 
"it'^grati.  i,  heuristic  approach  to  data  gathering,  data  transformation  through 
lu^rformance  prediction  and  data  display  has  been  satisfied.  For  organizations 
possessing  large  collec'ions  of  geographically  dispersed  homogeneous  computer 
systems,  this  potentially  permits  implementation  of  a centralized  approach  to 
perform  --.ce  analysis  and  system  design. 
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